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IMAGE  MODELING  AND  ENHANCEMENT  VIA  STRUCTURED  SPARSE  MODEL  SELECTION 
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ABSTRACT 

An  image  representation  framework  based  on  structured  sparse  model 
selection  is  introduced  in  this  work.  The  corresponding  modeling 
dictionary  is  comprised  of  a  family  of  learned  orthogonal  bases.  For 
an  image  patch,  a  model  is  first  selected  from  this  dictionary  through 
linear  approximation  in  a  best  basis,  and  the  signal  estimation  is  then 
calculated  with  the  selected  model.  The  model  selection  leads  to  a 
guaranteed  near  optimal  denoising  estimator.  The  degree  of  free¬ 
dom  in  the  model  selection  is  equal  to  the  number  of  the  bases,  typ¬ 
ically  about  10  for  natural  images,  and  is  significantly  lower  than 
with  traditional  overcomplete  dictionary  approaches,  stabilizing  the 
representation.  For  an  image  patch  of  size  \/N  x  ^/N,  the  computa¬ 
tional  complexity  of  the  proposed  framework  is  typically  2 

to  3  orders  of  magnitude  faster  than  estimation  in  an  overcomplete 
dictionary.  The  orthogonal  bases  are  adapted  to  the  image  of  interest 
and  are  computed  with  a  simple  and  fast  procedure.  State-of-the-art 
results  are  shown  in  image  denoising,  deblurring,  and  inpainting. 

Index  Terms —  Model  selection,  structured  sparsity,  best  basis, 
denoising,  deblurring,  inpainting 

1.  INTRODUCTION 

Image  enhancement  problems,  such  as  denoising,  deblurring  and 
inpainting,  are  typical  and  important  tasks  in  image  processing.  Es¬ 
timation  in  sparse  representations  provides  a  powerful  tool  for  such 
image  enhancement  tasks.  State-of-the-art  image  enhancement  re¬ 
sults  have  been  recently  obtained  with  learned  overcomplete  dictio¬ 
naries  and  sparse  coding  over  them  Eisiini.  In  these  approaches, 
overcomplete  local  dictionaries  are  learned  from  the  images,  and  the 
image  enhancement  is  performed  through  estimation  (sparse  cod¬ 
ing)  in  the  resulting  overcomplete  sparse  image  representations. 

Sparse  signal  estimation  in  an  overcomplete  dictionary  is  sub¬ 
ject  to  an  exponentially  large  degree  of  freedom.  Calculating  such 
estimation  is  not  only  computationally  demanding,  the  resulting  es¬ 
timate  is  also  likely  to  be  unstable  OUl,  which  may  deteriorate  the 
image  enhancement  results  ITSll . 

In  this  paper,  we  introduce  a  novel  image  representation  frame¬ 
work,  namely  structured  sparse  model  selection  (SSMS).  The  dic¬ 
tionary  is  comprised  of  a  family  of  learned  orthogonal  bases.  For  a 
given  image  patch,  a  model  is  selected  through  linear  approximation 
in  a  best  basis,  over  which  the  signal  estimation  is  then  calculated. 
The  model  selection  leads  to  a  guaranteed  near  optimal  denoising 
estimator.  The  degree  of  freedom  in  the  model  selection  is  equal  to 
the  number  of  the  bases,  typically  about  10  for  natural  images,  sig¬ 
nificantly  reduced  with  respect  to  the  traditional  overcomplete  dic¬ 
tionary  approaches,  and  thereby  stabilizing  the  representation.  For 
an  image  patch  of  size  \/N  x  \/N,  signal  estimation  with  SSMS  is 
calculated  with  a  complexity  ^{KMN),  where  K  and  M  are  respec¬ 
tively  the  number  of  the  bases  and  the  linear  approximation  dimen¬ 
sion,  whose  typical  values  lead  to  a  complexity  in  2  to  3 

orders  of  magnitude  faster  than  estimation  in  an  overcomplete  dic¬ 
tionary.  The  orthogonal  bases  are  adapted  to  the  image  of  interest 
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and  are  computed  with  a  simple  and  fast  procedure.  State-of-the-art 
results  are  shown  in  image  denoising,  deblurring  and  inpainting. 

2.  STATE-OF-THE-ART  ON  IMAGE  ENHANCEMENT 
State-of-the-art  image  enhancement  is  obtained  by  sparse  coding 
image  patches  with  learned  overcomplete  dictionaries  |[6l[T4l.  Im¬ 
ages  are  decomposed  into  overlapping  patches  /  G  where  a/A  x 
a/A  is  the  patch  size.  A  patch  is  assumed  to  be  sparsely  represented 
in  an  overcomplete  dictionary  ^  with  |r|  >  A: 

/  =  /a  +  wa  with  /a=  ^  (1) 

meA 

where  the  approximation  error  ||wa||2  «  ||/||2  and  |A|  <c  |r|.  A 
sparse  approximation  /  =  of  /  is  calculated  with,  for 

example,  an  orthogonal  matching  pursuit  that  approximates  an  f 
solution  na  or  with  an  minimization  that  calculates 

d  =  argmin||/-  Y  a[m]^m\\^  +  X\\a\\i.  (2) 

°  mer 

Given  an  image  patch  data  set  with  /  ^  A,  the  dic¬ 

tionary  ^  =  {0m}mer  is  learned  by  solving  the  following  mini¬ 
mization,  so  that  it  provides  sparse  representations  for  all  the  image 
patches,  / 

r.  ,  +  (3) 

i=l  rneT 

The  minimization  ([3])  is  typically  non-convex  and  a  local  solution  is 
calculated  with  an  iterative  procedure,  e.g.,  the  K-SVD  ifTl. 
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Fig.  1.  Sparse  signal  models.  Left:  the  overcomplete  dictionary  model.  In 
this  model  any  sparse  set  of  atoms  (columns  in  gray)  can  be  selected.  Right: 
the  structured  sparse  model  selection  model.  In  this  model,  the  dictionary 
is  comprised  of  a  family  of  orthogonal  bases,  only  one  selected  at  a  time. 
The  model  selection  is  calculated  with  linear  approximation  in  a  best  basis, 
i.e.,  only  the  first  few  atoms  of  one  basis  (in  gray)  can  be  selected. 

3.  STRUCTURED  SPARSE  MODEL  SELECTION 

The  underlying  signal  model  of  the  overcomplete  dictionary  ap¬ 
proach,  as  depicted  in  Fig.[Tl  is  that  an  image  patch  /  G  can  be 
accurately  approximated  with  a  few  atoms  {0m}meA  in  an  overcom¬ 
plete  dictionary  St  =  with  |r|  >N^  |A|.  To  approximate 

'|r| 

JA| 

selecting  the  atoms  is  exponentially  large  10^"^  for  typical  val¬ 
ues  A  =  64,  |r|  =256,  |A|  =  8),  which  may  lead  to  unstable  signal 
estimation  II.  It  has  been  shown  that  the  d  minimization  achieves 
more  stable  recovery  of  the  support  A  than  f  ifT^ .  However,  reliable 
signal  estimation  requires  further  stabilization  (IKTSl.  Attempting 
to  exploit  all  the  possibilities,  the  approximation  o  is  computed 
via  an  iterative  d  minimization  procedure  (O,  that  has  a  relatively 
high  computational  complexity,  ^(L|r|A),  where  L  is  the  number 
of  iterations  typically  in  the  order  of  10^  to  10^. 

Putting  structure  in  sparsity  stabilizes  the  estimation  Ii.  In  this 
section,  we  introduce  a  structured  sparse  model  selection  (SSMS) 
signal  model  that  reduces  tremendously  the  degree  of  freedom  as 
well  as  the  computational  complexity  in  signal  estimation,  and  on 
the  top  of  that,  provides  state-of-the-art  image  enhancement  results. 


3.1.  Structured  sparse  model 

As  illustrated  in  Fig.[Tl  SSMS  is  defined  with  a  dictionary  comprised 
of  K  sub-dictionaries  =  {^^}\<k<K^  each  =  {(\>m}\<m<N  be¬ 
ing  an  orthogonal  basis.  An  image  patch  /  G  is  assumed  to  be 
accurately  approximated  with  a  linear  approximation  in  a  best  or¬ 
thogonal  basis  indexed  by 

M 

f  =  fM+  with  =  £  (/,  )  (j)j^ ,  (4) 

m=l 

where  the  linear  approximation  error  ||  wjj  |p  <C  ||/|p  and  M  <^N  is 
the  linear  approximation  dimension. 

Compared  with  the  overcomplete  dictionary  model  (IB,  the  SSMS 
model  (|4l(  is  much  more  structured  and  the  structure  comes  from 
two  concepts.  First,  the  estimation  in  the  best  orthogonal  basis  ex¬ 
cludes  the  possibility  to  select  atoms  across  the  bases,  which  shrinks 

the  degree  of  freedom  in  atom  selection  from  ^  ^  to  ^  ^  ^  ^  . 

Second,  the  linear  approximation  in  each  basis  further  reduces  the 
degree  of  freedom  to  just  K,  stabilizing  the  encoding  and  reducing 
the  computational  cost. 

3.2.  Structured  model  selection 

The  best  basis  in  Q  is  computed  by  maximizing  the  projection 
energy  on  {0;^}i<m<M  over  all  the  orthogonal  bases  G 

M 

ko  =  arg  max  \\f^f  =  arg  max  £  | (/, (5) 

The  model  selection  ([5])  has  a  degree  of  freedom  K  and  is  calculated 
with  a  complexity  ^ (KMN) .  With  the  typical  values  A  =  64,  M  =  8, 
^  =  18  as  we  will  see  in  Section[33l  the  model  selection  complexity 
is  ^(A^). 

The  following  theorem  shows  that  when  the  true  original  signal 
is  contaminated  by  a  Gaussian  white  noise  w, 

y[n]  =  f[n]  +  w[^]  for  1  <  ^  <  A,  (6) 

the  empirical  model  selection  leads  to  a  near  optimal  denoising  es¬ 
timator  whose  risk  is  upper  bounded  by  4  times  the  best  estimation 
obtained  with  an  oracle. 

Theorem  1.  Let  be  the  noise  variance  and  T  =  Xa^/\ogJ^KW) 
with  A  >  y^32  +  .  For  any  f  G  the  empirical  model 

selection  ^ 

h  =  arg  max  =  arg  max  £  (7) 

k=l,...K 

yields  a  linear  projection  estimator 

M 

(8) 

m=l 

of  f,  which  satisfies 

E[\\f-ff]  +  (9) 

where  fh  =  (/, 

Theorem[T]is  derived  from  the  general  model  selection  result  of 
Barron,  Birge  and  Massart  O,  which  shows  that  the  projector  es¬ 
timator  over  the  best  empirical  set  in  an  overcomplete  dictionary  is 
near  optimal.  Assuming  linear  approximation  in  a  best  orthogonal 
basis,  the  structured  sparse  model  selection  (|7])  is  computed  calcu¬ 
lating  the  linear  projection  energy  in  each  basis,  and  enjoys  the  ad¬ 
vantage  of  being  free  of  the  thresholding  parameter  T,  on  contrast 
to  El. 


3.3.  Bases  design 

The  orthogonal  bases  {^^}i<k<K  are  initialized  with  Principal  Com¬ 
ponents  Analysis  (PC A)  over  synthetic  contour  images.  Based  on 
the  model  selection,  they  are  then  adapted  to  the  image  of  interest 
by  applying  PC  As  again  over  that  image. 

3.3.1.  Directional  bases  initialization 

Fig.  2.  The  first  8  atoms  in  with  Ok  =  30°. 

As  shown  in  ID,  the  most  prominent  atoms  in  a  learned  over¬ 
complete  dictionary  look  like  edge  patterns.  In  order  to  capture  di¬ 
rectional  patterns,  the  orthogonal  bases  in  SSMS  are  initialized  by 
computing  PCAs  over  synthetic  contour  images  of  various  orienta¬ 
tions,  one  basis  for  each  orientation.  At  the  orientation  %,  synthetic 
contours  with  different  amount  of  blur  are  created.  Image  patches 
of  size  \/N  X  a/A  touching  the  contours  at  different  lo¬ 
cations  are  collected.  The  orthogonal  basis  is  obtained  by  calcu¬ 
lating  PC  A  over  these  patches. 

The  eigenvalues  of  the  dictionary  atoms  decrease  rapidly  and 
become  negligible  after  about  the  s/N-th  atom.  As  shown  in  Fig.[2l 
the  first  about  s/N  =  8  atoms  oscillate  in  the  direction  orthogonal  to 
Ok  from  low-  to  high-frequency  and  are  quasi-constant  in  Ok.  Other 
atoms  that  complete  the  orthogonal  basis  look  unmeaningful.  The 
orientation  Ok  is  uniformly  sampled  from  0°  to  180°,  typically  with 
a  step  of  10°,  which  results  in  ^  =  18  orthogonal  bases. 

In  order  to  better  capture  more  complex  texture  patterns,  a  DCT 
basis  is  added  to  the  dictionary 

3.3.2.  Bases  adapted  to  image 

The  bases  are  adapted  to  the  image  of  interest  by  applying  the  PCAs 
over  the  image  patches,  grouped  following  the  model  selection.  The 
structured  model  selection  with  the  directional  bases  introduced  above 
assigns  a  model  k  to  each  patch.  The  basis  with  1  <  k  <  ^,  is 
adapted  to  the  image  by  recalculating  the  PCA  with  all  the  image 
patches  that  have  been  assigned  to  the  model  k. 

3.4.  Best  basis  estimation 

The  signal  estimation  is  calculated  with  a  non-linear  thresholding 
estimator  in  the  best  basis 

(10) 

mGA 

where  A  =  {m  :  |  (y,  )  |  >  T}.  In  effect,  the  non-linear  estimation 

typically  improves  the  estimate  with  respect  to  the  linear  estima¬ 
tor  ii),  as  the  former  is  more  robust  to  the  signal  model  error. 

4.  EXPERIMENTAL  RESULTS  &  CONCLUSIONS 

4.1.  Denoising 

4.1.1.  Gaussian  white  noise  removal 

We  first  tested  on  images  corrupted  with  Gaussian  white  noise  of 
variance  G^.  The  SSMS  denoising  is  conducted  with  patches  of  size 
8  X  8,  M  =  8,  =  18  and  T  =  3(7.  Table  [H  summarizes  its  per¬ 

formance  in  comparison  with  the  learned  overcomplete  dictionary 
(LCD)  El,  the  Non-Local  Means  (NLmeans)  n,  and  the  Gaus¬ 
sian  Scale  Mixtures  (GSM)  IITtI.  The  results  of  LCD  and  GSM  are 
quoted  from  El  El.  and  those  of  NLmeans  are  produced  with  the 
code  from  Buades  mi-  SSMS,  with  a  much  smaller  computational 
complexity,  generates  the  highest  PSNRs  in  most  cases.  Its  gain  on 
Barbara,  rich  in  geometrical  texture,  is  particularly  significant.  An 
example  illustrated  in  Fig.  El  shows  that  the  SSMS  denoising  restores 
some  very  fine  patterns. _ 

^The  experimental  results  are  available  at 

http://www.cmap.polytechnique.fr/^yu/research/ASIFT/demo.html. 
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Table  1.  Denoising  performance  (in  PSNR).  Top  left:  NLmeans  Top 
right:  GSM  Bottom  left:  learned  overcomplete  dictionary  Bottom 
right:  the  proposed  SSMS  denoising.  In  each  set  the  best  result  is  in  bold. 


Original 


Noisy  28.09  dB 


SSMS  denosing  34.56  dB 


Fig.  3.  Denoising.  From  left  to  right:  a  cropped  region  from  Lena,  noisy 
version  (g  =  lOj ,  the  proposed  SSMS  denoising. 


4.1.2.  Gaussian  colored  noise  removal  —  deblurring 

In  these  experiments,  the  clean  image  /  is  blurred  and  contaminated 

by  a  Gaussian  white  noise  w 

y[n]  =  {f  ^  h)[n]  ^  w[n]  for  l<n<N^  (11) 

where  y  is  the  observed  image  and  h  is  the  blurring  kernel  that  is 
assumed  to  be  known.  Deblurring  aims  at  estimating  /  from  y. 

The  deblurring  can  be  casted  to  a  denoising  problem  by  first 
making  a  pseudo-inverse  of  h  in  the  Fourier  domain  with  a  Wiener 
filter,  and  then  removing  the  Gaussian  colored  noise  amplified  by 
the  Wiener  filtering  . 

The  SSMS  deblurring  is  compared  with  the  popular  ForWaRD 
algorithm  ifT^.  The  same  Wiener  filter  is  applied  in  both  meth¬ 
ods.  While  the  proposed  SSMS  deblurring  proceeds  the  denoising 
with  the  SSMS  model,  ForWaRD  denoises  by  translation-invariant 
wavelet  shrinkage.  A  Monte  Carlo  procedure  is  applied  for  both 
methods  to  estimate  the  Gaussian  colored  noise  variance  on  each 
coefficient.  As  summarized  in  Table  [2l  the  SSMS  deblurring  im¬ 
proves  on  average  about  0.7  dB  in  PSNR  over  ForWaRD.  An  ex¬ 
ample  illustrated  in  Fig.  [4| shows  that  the  SSMS  deblurring  removes 
more  effectively  the  colored  noise  and  better  restores  fine  patterns. 
Since  the  two  methods  apply  the  same  Wiener  filter,  they  achieve 
the  same  amount  of  deblurring,  which  will  be  further  improved  by 
the  super-resolution  deblurring  described  in  Section  14.2.21 


Lena 

Cameram. 

House 

Boat 

1/5 

33.13 

33.95 

27.18 

27.78 

32.35 

33.13 

29.98 

30.66 

33.18 

34.21 

27.09 

27.96 

32.31 

33.65 

30.22 

30.83 

2/5 

31.17 

31.95 

24.51 

24.63 

30.31 

30.87 

27.84 

28.22 

31.31 

32.11 

25.01 

25.02 

30.99 

31.95 

28.23 

28.38 

Table  2.  Deblurring  performance  (in  PSNR).  The  left  column  specifies  the 
std  of  the  Gaussian  blur  kernel  Gjy  and  of  the  Gaussian  white  noise  Gn-  Top 
left:  ForWaRD  l[T6\l.  Top  right:  the  proposed  SSMS  deblurring  (Sec\4.1.2\. 
Bottom  left:  DSD  177]/.  Bottom  right:  the  proposed  SSMS  super-resolution 
deblurring  (Sec\4.2.2\.  In  each  set  the  best  result  is  in  bold. 


4.2.  Inverse  Problems 

Image  enhancement  often  requires  to  solve  an  inverse  problem,  which 
amounts  to  estimate  an  image  /  in  a  space  of  larger  dimension  than 


Original  Blurred,  noisy  20.61  dB  ForWaRD  20.69  dB  SSMS  21.34  dB 

Fig.  4.  Deblurring.  From  left  to  right:  a  cropped  region  from  Barbara, 
blurred  and  noisy  version,  ForWaRD  the  proposed  SSMS  deblurring. 
that  of  the  degraded  image  y  obtained  through  a  linear  operator  U, 
and  further  contaminated  by  an  additive  noise  w: 

y[n]  =  U  f[n]  F  w[n]  for  \<n<N.  (12) 

The  deblurring  addressed  in  SectionlT.1.2lis  a  classical  inverse  prob¬ 
lem,  where  /7  is  a  convolution  operator. 

Given  a  dictionary  ^  =  {0m}mer  that  provides  a  sparse  rep¬ 
resentation  for  the  original  image  /  ([T]),  the  degraded  image  y  is 
sparsely  represented  in  the  transformed  dictionary  =  {U 0m}mer- 

y  =  yA  +  w'  with  )'A=  ^  a[m]Ui^m,  (13) 

mGA 

where  w'  =  I/wa  -f-  w  and  1 1  w'  | p  <C  | |y  | P .  The  sparse  super-resolution 
approach  (zuni  seeks  to  estimate  the  coefficients  a  and  the  support 
A  =  {m  :  d[m]  7^  0}  of  /  from  the  degraded  signal  y: 

a  =  argmin||)'-  V  a[m]U +  M\a\\i.  (14) 

meT 

The  resulting  sparse  super-resolution  estimate  of  /  is 

f  =  Y,  a[m\^m-  (15) 

mGA 

In  addition  to  the  sparse  representation  requirement  on  the 
following  conditions,  not  addressed  in  ||7|  [TTl,  are  necessary  for  a 
high-quality  super-resolution  estimate  /  ca: 

•  Recoverability.  >  0,  Vm  G  A. 

•  Stability,  0m}mer  should  be  incoherent  enough. 

While  the  first  condition  suggests  the  recoverability  of  the  support 
A  from  y,  the  second  condition  implies  the  stability  of  the  support 
estimate.  It  has  recently  been  shown  that  imposing  structure  in  the 
estimation  essentially  stabilizes  the  estimate  Elia. 

The  SSMS  model  enjoys  the  advantage  of  being  highly  struc¬ 
tured.  Instead  of  solving  (O,  SSMS  calculates  for  each  G  & 

M 

=  argmin^^  =  argmin||y  —  'V  a[m]U^^\\^  ^ X\\a\\\.  (16) 

The  best  basis  is  obtained  with 

k{\  =  arg  min  e^ .  (17) 

k=l,...,K 

The  sparse  super-resolution  (fTH)  and  (flSl)  is  then  calculated  with  the 
best  basis  . 

4.2.1.  Super-resolution  inpainting 

Inpainting  aims  at  estimating  /  from  y  in  (O,  where  /7  is  a  binary 
random  mask  on  the  image  and  w  is  typically  neglected.  The  SSMS 
inpainting  is  calculated  with  patches  of  size  8  x  8,  M  =  8  and  K  = 
18.  The  minimization  is  calculated  via  a  weighted  Lasso  with 
the  LARS  algorithm  (121 .  Table [3] summarizes  the  SSMS  inpainting 
results,  in  comparison  with  the  EM  (91  and  the  MCA  (71  algorithms 
that  utilize  a  dictionary  comprised  of  a  curvelet  frame  and  a  local 
DCT,  as  well  as  with  (TOl  that  utilizes  a  local  DCT.  The  results  of 
MCA  and  EM  are  produced  with  the  authors’  softwares  QISl.  d 
is  a  degenerated  case  of  SSMS  consisting  of  only  one  basis  that  is 
the  DCT.  The  SSMS  inpainting  generates  the  highest  PSNRs  in  all 
cases.  When  the  masking  percentage  becomes  important  at  60%,  the 
highly  structured  SSMS  model  significantly  outperforms  the  second 
best  algorithm  by  more  than  2  dB  in  PSNR.  As  illustrated  in  Fig  [5l 
the  SSMS  inpainting  accurately  restores  the  image  geometry  from  a 
very  heavily  masked  image. 


masked/PSNR 

Lena 

Barbara 

House 

Boat 

30%/10.14 

40.32 

33.40 

38.56 

32.46 

41.95 

33.57 

37.33 

31.34 

38.53 

40.76 

38.95 

40.70 

40.67 

43.00 

36.17 

37.69 

60%/7.11 

33.43 

29.91 

29.77 

28.52 

34.10 

29.56 

30.40 

27.58 

33.95 

36.02 

32.33 

34.50 

35.23 

37.41 

31.00 

32.53 

Table  3.  Inpainting  performance  (in  PSNR).  Top  left:  HUl.  Top  right: 
EM  Bottom  left:  MCA  Eil.  Bottom  right:  the  proposed  SSMS  inpaint¬ 
ing.  In  each  set  the  best  result  is  in  bold. 


Original  Masked  6.30  dB  SSMS  inpainting  36.63  dB 

Fig.  5.  Inpainting.  From  left  to  right:  a  cropped  region  from  Lena,  60% 
randomly  masked  version,  the  proposed  SSMS  inpainting. 


4.2.2.  Super-resolution  deblurring 

Super-resolution  deblurring  has  the  capacity  to  restore  image  pat¬ 
terns  at  frequencies  eliminated  by  the  convolution  operator  U,  which 
is  impossible  with  the  Wiener  filtering  approach  described  in  Sec¬ 
tion  [4X2l  The  SSMS  super-resolution  deblurring  is  calculated  with 
a  hierarchy  of  two  families  of  orthogonal  bases. 

The  directional  bases  introduced  in  Section  13.3.11  do  not  have 
super-resolution  deblurring  ability  since  they  do  not  satisfy  the  re¬ 
coverability  condition  described  above.  Indeed  these  oscillatory 
atoms,  as  illustrated  in  Fig.  [21  have  Dirac  supports  in  Fourier  and 
can  be  therefore  killed  after  the  convolution.  Following  the  hier¬ 
archical  sparse  decomposition  scheme  decoupling  orientation  and 
position  introduced  in  O,  a  family  of  orthogonal  bases  that  speci¬ 
fies  the  edge  position  in  the  patch  is  cascaded  with  each  directional 
basis  of  the  corresponding  orientation.  The  atoms  in  the  position 
bases  are  localized  around  the  underlying  position,  and  thus  have 
spread  spectrum  in  Fourier  as  required  by  the  recoverability  condi¬ 
tion.  The  position  bases  are  computed  with  the  PCA  procedure  in 
the  same  way  as  described  in  Section U. 3. 11  for  each  basis  the  train¬ 
ing  patches  being  extracted  at  a  fixed  position  from  the  synthetical 
contour  images.  Two  basis  selections  (fTTl)  are  cascaded  in  the  hi¬ 
erarchy:  the  first  selects  the  best  direction  basis  and,  given  the  best 
orientation,  the  second  selects  the  best  position  basis.  The  sparse 
super-resolution  deblurring  is  then  calculated  with  the  best  position 
basis  in  the  selected  direction. 

The  SSMS  super-resolution  deblurring  is  conducted  with  patches 
of  size  12  X  12,  M  =  20  and  ^  =  18.  The  minimization  is  calcu¬ 
lated  via  a  weighted  Lasso  with  the  LARS  algorithm  |[T2ll.  The  patch 
boundary  issue  is  treated  in  the  same  way  as  in  ifTll.  Tabled  sum¬ 
marizes  the  performance  of  the  SSMS  super-resolution  deblurring, 
in  comparison  with  ForWaRD,  the  SSMS  deblurring  (without  super¬ 
resolution),  and  the  direct  sparse  deconvolution  (DSD)  ifTTlPl  The 
SSMS  super-resolution  deblurring  generates  the  highest  PSNRs  in 
all  cases,  followed  by  the  SSMS  deblurring.  As  its  gain  with  respect 
to  the  latter  lies  essentially  on  edges  that  take  a  small  proportion  in 
the  image,  the  PSNR  improvement  over  the  whole  image  is  not  sig¬ 
nificant.  However,  the  visual  benefit  is  substantial.  As  illustrated  in 

^We  thank  very  much  Yifei  Lou  for  kindly  making  the  DSD  experiments. 


Fig.  [6l  the  super-resolution  deblurring  restores  sharper  edges  than 
the  SSMS  deblurring  without  super-resolution  described  in  14.1. 21 


Fig.  6.  Deblurring.  From  left  to  right:  a  cropped  region  from  Cameraman, 
blurred  and  noisy  version,  the  proposed  SSMS  deblurring  (without  super¬ 
resolution)  and  the  proposed  SSMS  super-resolution  deblurring. 

To  conclude,  let  us  mention  that  we  are  currently  extending  this 
work  to  color  image  modeling  and  enhancement. 
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